Natural head motion synthesis driven by acoustic prosodic features

Natural head motion is important to realistic facial animation and engaging human-computer interactions. In this paper, we present a novel data-driven approach to synthesize appropriate head motion by sampling from trained Hidden Markov Models (HMMs). First, while an actress recited a corpus specifically designed to elicit various emotions, her 3D head motion was captured and further processed to construct a head motion database that included synchronized speech information. Then, an HMM for each discrete head motion representation (derived directly from data using vector quantization) was created by using acoustic prosodic features derived from speech. Finally, first order Markov models and interpolation techniques were used to smooth the synthesized sequence. Our comparison experiments and novel synthesis results show that synthesized head motions follow the temporal dynamic behavior of real human subjects.